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Abstract 

The  Air  and  Space  Operations  Center  (AOC)1  Command  and  Control 
data  center  node  is  the  senior  air  power  element  on  any  battlefield.  The 
AOC  provides  the  Commander  the  capability  to  plan  and  execute  theater¬ 
wide  air  and  space  operations.  Two  primary  documents  are  created  daily: 
the  Air  Tasking  Order  (ATO)  and  the  Airspace  Control  Order  (AGO). 


In  any  operation  involving  air  power,  usually  a  single  commander  is 
responsible  for  the  air  power  force.  In  a  theater-size  military  campaign, 
as  many  as  2500  people  inside  a  Combined  Air  Operations  Center 
(CAOC)  move  massive  amounts  of  information  across  multiple  networks. 
The  CAOC  provides  the  Commander  the  capability  to  direct  and  super¬ 
vise  activities  of  assigned  or  attached  forces  and  monitor  the  actions  of 
both  enemy  and  friendly  forces.  The  ATO  and  AGO  promulgate  his  will 
throughout  his  command. 


Advanced  optimization  methods  and  statistical  sampling  techniques 
may  significantly  help  quantitatively  model  and  understand  the  interac¬ 
tion  of  combatants.  This  article  explores  the  use  of  Response  Surface 
Methodology  (RSM)  as  an  approach  to  understand  and  optimize  opera¬ 
tional  air  power  and  illustrates  its  application  using  an  operational  train¬ 
ing  system  in  conjunction  with  a  fictitious  force-on-force  scenario. 


1 .  For  the  purpose  of  this  article,  the  terms  CAOC  and  AOC  are  synonymous. 
The  Joint  Force  Air  Component  Commander  JFACC)  is  the  senior  airpower 
leader;  the  CAOC/AOC  is  his  supporting  staff  and/or  their  physical  location. 
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Introduction 

Humans  are  notoriously  bad  at  visualizing  any  mathematical  rela¬ 
tionship  beyond  a  direct  or  proportional  linkage.  Modern  techniques 
help  eliminate  this  condition.  A  plumb  bob  is  an  ancient  tool  used  to 
create  the  Egyptian  pyramids  and,  if  one  digs  deep  into  most  plumb¬ 
ers  or  carpenters’  toolboxes,  one  can  find  that  tool  today,  modified, 
but  recognizable  to  any  ancient  Egyptian  construction  foreman. 
Command  and  Control  (C2)  has  been  around  at  least  as  long  as 
plumb  bobs,  but  using  the  same  tools  the  ancients  used  does  not 
guarantee  success.  When  one  thinks  of  the  best  “ancient”  air  power 
commanders  it  is  easy  to  envision  the  “old-fashioned”  fighter  pilot: 
a  natural  leader  and  an  intuitive  tactician  leading  his  command  to 
victory. 

The  Wright  brothers  advanced  aviation  not  by  improving  the  under¬ 
standing  of  lift,  but  by  mastering  the  interactions  of  control.  During 
the  Vietnam  War,  flying  F-4  Phantoms  or  F-105  Thunderchiefs 
was  dangerous  work.  Col.  Robin  Olds,  the  commander  of  the  8th 
Tactical  Fighter  Wing,  came  up  with  the  qualitative  strategy  of  lur¬ 
ing  North  Vietnam’s  MiG-2  Is  into  battle  with  F-4s  masquerading  as 
the  more  vulnerable  F-105s  (Scutts  1988).  The  operation  was  named 
Bob.  It  required  a  massive  Air  Force-wide  effort  to  bring  it  to  frui¬ 
tion.  The  battle  was  a  total  success.  Thirty-plus  years  later,  we  still 
qualitatively  create  air  battle  plans.  The  question  explored  is  “Can 
the  science  of  control  be  used  to  help  quantitatively  understand  and 
optimize  the  interaction  of  air  combatants?”  Sun  Tzu  said:  “If  you 
know  the  enemy  and  know  yourself  you  need  not  fear  the  results  of  a 
hundred  battles”  (Giles  1910).  Therefore,  the  goal  of  this  article  is  to 
explore  whether  the  use  of  a  strict  quantitative  technique  has  viabil¬ 
ity  when  used  on  a  problem  set  as  complex  as  combat  air  power. 
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What  is  an  AOC? 

Today,  the  Air  and  Space  Operations  Center  (AOC)2  Weapon 
System,  AN/USQ;163  Falconer,  military  C2  data  center  node  is  the 
senior  element  of  the  Theater  Air  Control  System  on  the  battlefield 
(Kometer  2005).  The  primary  function  of  the  divisions  of  the  AOC 
is  to  produce  and  execute  an  Air  Tasking  Order  (ATO)  and  associ¬ 
ated  documents  like  the  Airspace  Control  Order  (ACO).  The  Air 
Force  fielded  five  permanent  Falconers  worldwide  to  meet  continu¬ 
ing  air  power  challenges.  In  any  operation  involving  air  power,  a 
single  commander  is  designated  the  responsible  member  for  all  air 
power  forces  assigned  and  attached.  In  a  theater-size  military  cam¬ 
paign,  as  many  as  2500  people  inside  the  Combined  AOC  (CAOC) 
move  massive  amounts  of  information  across  multiple  communica¬ 
tion  networks  of  various  security  levels.  The  Falconer  is  the  core 
production  hub  of  the  much  larger  CAOC  facility/ compound.  The 
entire  CAOC  provides  the  Commander  the  capability  to  direct  the 
activities  of  assigned,  supporting,  or  attached  forces  and  monitor  the 
actions  of  both  enemy  and  friendly  forces.  Walking  into  any  of  the 
five  worldwide  CAOCs  for  the  first  time  is  an  extraordinary  expe¬ 
rience.  It  is  just  what  you  expect  of  the  nerve  center  of  the  most 
powerful  Air  Force  on  earth.  Huge  projection  screens  show  the  exact 
location  of  every  military  aircraft  flying  over  the  theater  of  opera¬ 
tions;  CNN,  Fox,  and  other  news  organizations;  and  other  situation 
displays.  Rows  of  professional  warriors  operate  computer  screens 
and  banks  of  telephones  communicating  worldwide  while  absorbing 
vast  amounts  of  information  from  organizations  across  the  planet. 
The  Combined  Force  Air  Component  Commander  (CFACC)  sits 
in  a  room  with  his  key  staff.  Video  and  data  screens  show  live  feeds 
from  various  sensors  over  the  battlefield.  Chat  rooms  on  computer 
screens  exchange  information  across  all  security  levels.  An  interest- 


2.  For  the  purpose  of  this  article,  Joint  Force  Air  Component  Commander 
JFACC)  and  Combined  Force  Air  Component  Commander  (CFACC)  are 
synonymous. 
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ing  question  to  be  asked  is  whether  these  modern  warriors’  efforts 
can  be  changed  from  their  qualitative  approach  and  augmented 
with  quantitative  tools? 


What  is  an  ATO? 

With  all  this  command  and  control  technology,  an  assertion  is  that 
the  most  critical  weapon  of  war  is  the  human  mind;  the  rest  can 
be  viewed  as  just  tools.  To  understand  complex  processes  such  as 
air  war,  it  may  sometimes  be  best  to  combine  all  the  variables  into 
simplified  models  that  can  represent  their  interactions.  The  ATO 
and  ACO  are  the  documents  used  to  disseminate  the  commander’s 
plan  for  all  combat  air  power  forces.  For  an  aviator,  these  are  the 
only  two  documents  provided  by  higher  headquarters  to  answer  the 
question,  what  am  I  doing  tomorrow?  The  ATO  and  ACO  are  United 
States  Message  Text  Format  (USMTF)  military  messages  that  pro¬ 
vide  a  written  description  of  the  next  day’s  air  battle  plan.  Based 
on  experience,  the  goal  of  building  the  ATO /ACO  is  to  provide  a 
single  source  document  for  everything  that  flies  and  provide  aware¬ 
ness  to  other  combatants  in  the  Area  of  Responsibility  (AOR),  and 
anything  that  uses  airspace  within  the  AOR,  in  the  next  24-hour 
period.  These  documents  may  be  hundreds  of  pages  of  computer 
printout  traditionally  approved  and  transmitted  12  hours  before 
execution.  “In  the  AOC,  two  separate  networks  exist  and  there  are 
limited  touch  points.  One  is  a  technical  network  that  conveys  data 
and  the  other  is  a  human  command  network  that  analyzes  or  syn¬ 
thesizes  that  data  transforms  it  to  information,  and  produces  deci¬ 
sions  that  result  in  output.  These  independent  networks  converge 
at  the  individual  personal  computer  (PC).  In  a  combat  AOC,  there 
are  generally  three  to  ten  computers  at  each  person’s  workspace” 
(Simpson  2008).  As  the  technical  infrastructure  moves  to  service  ori¬ 
ented  architecture  (SOA),  there  will  be  little  change  to  human  driven 
constraints  dictated  in  building  an  ATO. 
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There  is  a  12-hour  period  between  publication  of  the  ATO  and 
the  start  of  the  battle  plan.  As  soon  as  the  tactical  units  receive  the 
ATO /AC  O,  they  begin  detailed  planning  to  create  mission-planning 
folders  for  the  aircrews.  At  the  same  time,  maintenance  receives  the 
ATO,  builds  the  ordnance  required,  and  starts  loading  the  aircraft. 
The  time  between  D-12  and  D+0  is  critical  for  these  functions  to 
accrue.  At  D+0  hours,  execution  of  the  ATO  and  implementation 
of  the  ACO  occurs.  The  CAOC  makes  adjustments  as  the  battle 
unfolds.  These  adjustments  can  be  minor,  such  as  approving  a  Time 
on  Target  (ToT)  change  by  1 5  minutes  due  to  a  wing  maintenance 
delay,  or  so  major  that  every  line  of  the  ATO  is  rendered  invalid  and 
the  order  must  be  recreated.  It  is  very  hard  for  any  Commander  to 
visualize  and  optimize  the  interactions  of  all  these  moving  parts. 


Understanding  Interactions:  An  Approach 

The  AOC  is  an  organization  that,  on  its  best  days,  is  qualitatively 
efficient  and  accurate  in  its  planning  and  execution.  A  qualitative 
approach  augmented  with  quantitative  techniques  may  have  the 
potential  to  improve  the  efficiency,  accuracy,  and  specificity  required 
in  the  operational  planning  and  tactical  delivery  of  air  power.  If 
one  considers  an  Air  Battle  Plan  as  a  large-scale  black  box  of  inter¬ 
actions,  it  is  easier  to  comprehend  than  all  the  individual  moving 
actions.  Tanker  and  other  support  aircraft  become  binding  agents 
and  weapons  and  enemy  actions  become  catalysts  of  change.  From 
this  perspective,  system  parameter  design  techniques  using  RSM 
can  be  applied  to  quantitatively  model  and  study  operations.  System 
design  is  the  process  of  applying  knowledge  to  produce  a  basic  func¬ 
tional  design  and,  in  this  case,  it  would  produce  a  qualitatively  cre¬ 
ated  ATO.  The  original  ATO  created  by  the  AOC  would  define  the 
attributes  of  the  Air  Battle  Plan  undergoing  analysis.  Assuming  zero 
transportation  time,  the  maximum  analysis  time  would  only  be  1 2 
hours.  The  qualitative  initial  design  may  be  functional,  but  it  may 
be  far  from  optimal,  in  terms  not  easily  visualized  by  experts  creat¬ 
ing  it.  The  objective  in  parameter  design  is  to  identify  the  settings 
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that  optimize  the  desired  performance  characteristic  (Phadke  1989; 
Kackar  1985).  We  often  see  Subject  Matter  Experts  (SMEs)  in  the 
AOC  working  qualitatively  until  the  plan  “looks  good”  or  they  run 
out  of  time  to  do  anything  different.  Experimenting  with  the  design 
variables  either  one  at  a  time  or  by  trial  and  error  is  a  common 
approach  to  optimization  (Phadke  1989;  Bendel  1988).  Elowever, 
this  approach  can  lead  to  a  very  long  and  expensive  time  span  for 
completing  the  design.  Furthermore,  when  using  a  one  variable  at  a 
time  approach,  parameter  interactions  that  may  affect  the  optimum 
results  may  not  be  identified  (Gunter  1990).  The  result  in  most  cases 
is  a  product  design  (ATO)  that  may  be  far  from  the  most  advanta¬ 
geous.  To  determine  the  optimum  conditions,  a full factorial  approach 
in  which  all  possible  combinations  of  parameter  values  are  tried 
may  be  necessary.  Elowever,  it  must  be  noted  that  as  the  number  of 
parameters  studied  increased,  a  full  factorial  approach  quickly  grows 
exponentially  large,  e.g.,  13  factors  at  three  levels  would  require 
studying  1,594,323  (313)  experiments. 


Understanding  Interactions:  Assumptions 

To  validate  that  a  quantitative  approach  is  possible  to  study  improv¬ 
ing  an  ATO,  the  factors  are  limited  to  four  different  types  of  aircraft 
assigned  to  various  units  at  two  different  levels  (full  up  and  30% 
reduced).  It  is  assumed  the  aircraft  would  have  forward-firing  mis¬ 
sile  ordnance,  require  air  refueling,  and  go  well  beyond  the  forward 
edge  of  the  battle  area  or  be  purely  defensive  air-to-air  aircraft.  The 
model  used  to  simulate  combat  was  an  operational  training  tool 
named  Command  and  Control  Weapon  System  Part  Task  Trainer 
(C2WSPTT)  (pronounced  chew-spit)  used  to  fly  out  ATOs  and 
simulate  combat  to  an  AOC  training  audience.  C2WSPTT  was  the 
only  model  readily  available  to  accomplish  the  necessary  data  runs. 
The  scenario  selected  was  an  unclassified  training  scenario  previ¬ 
ously  created  to  provide  AOC  operators  basic  training.  To  make  the 
unclassified  scenario  robust,  we  used  a  training  scenario  that  had  63 1 
pieces  of  airspace,  550  blue  (friendly)  missions,  and  197  red  (enemy) 
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missions.  Consequently,  if  the  ATO  were  run,  it  would  complete  24 
hours  of  missions  in  approximately  25  minutes.  Sixteen  experimen¬ 
tal  test  runs  would  require  a  little  more  than  half  of  the  1 2  hours  tra¬ 
ditionally  available.  The  reported  speed  is  65  times  normal.  When 
a  real  ATO  is  flown  with  all  the  factors  associated  with  combat,  the 
maximum  speed  of  C2WSPTT  is  about  2.5  times  normal  operating 
on  commercially  available  computer  platforms.  To  ensure  no  air¬ 
craft  was  shot  down  from  ground  by  Surface  to  Air  Missiles  (SAMs), 
we  turned  them  all  (both  red  and  blue)  off,  as  the  SAM  factor  would 
have  overwhelmed  the  number  of  red  aircraft  destroyed.  Normally, 
Intelligence  will  brief  two  scenarios  for  enemy  actions,  most  likely  and 
worst  case.  We  were  only  able  to  create  one  red  ATO;  therefore,  it 
was  flown  against  the  most  likely  scenario  only.  Without  knowing  the 
conceptual  foundation  or  algorithms  of  C2WSPTT,  potential  sto¬ 
chastic  variation  in  ATO  fly  out  was  minimized  by  elimination  of 
the  same  mission  numbers  in  the  ATO  whenever  the  30%  reduction 
was  required. 

With  the  parameters  of  the  simulation  in  place,  we  were  ready  to 
explore  if  an  engineering  quantitative  method  may  be  used  to  opti¬ 
mize  an  ATO.  RSM  is  a  set  of  mathematical  and  statistical  tech¬ 
niques  for  analysis  designed  to  create  a  mathematical  model  to  effi¬ 
ciently  explore  the  effects  of  a  set  of  parameters.  Using  RSM  in  a 
military  setting  is  a  valid  experiment  as  defined  by  the  Code  of  Best 
Practice  for  Experimentation  (Alberts  et  al.  2002).  In  the  case  of  com¬ 
bat  air  power,  the  variables  were  number  of  blue  aircraft  lost  (mini¬ 
mized)  and  number  of  red  aircraft  destroyed  (maximized).  Using 
experimental  design  methods,  RSM  seeks  to  relate  a  response  or  output 
characterized  to  the  values  of  a  number  of  predictors  or  input  vari¬ 
ables  that  affect  it  (Box  and  Draper  1987).  Response  can  be  defined 
as  the  performance  or  quality  characteristic  of  interest  (e.g.,  yield, 
weight,  number  of  aircraft).  These  techniques,  introduced  by  Box 
and  Wilson  (Myers  1971)  and  later  expanded  by  others,  consist  of 
designing  the  experiment  and  the  subsequent  analysis  of  experi¬ 
mental  data  (Cornell  1990).  RSM  may  lead  to  a  rapid  and  efficient 
exploration  of  the  ATO  and  to  estimated  optimum  conditions  within 
limited  time  and  experimental  data. 
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Steps  Involved  in  Parameter  Design 

There  are  six  steps  in  a  typical  parameter  design  (Phadke  1989): 

a.  Identify  the  characteristic  to  be  observed, 

b.  Identify  the  factors  and  levels, 

c.  Define  the  most  likely  interactions  between  parameters 
(factors), 

d.  Design  the  matrix  experiment  required  and  define  the  data 
analysis  plan, 

e.  Conduct  the  experiments, 

f.  Analyze  the  data  to  determine  optimum  levels  of  factors. 

The  first  four  steps  are  required  for  planning  the  experiment.  In  the 
fifth  step,  the  experiments  are  conducted.  In  this  case,  the  operational 
training  system  is  run.  In  step  six,  experimental  results  are  analyzed, 
optimum  levels  are  determined,  and  a  confirmation  experiment  is 
conducted  to  verify  results  (i.e.,  if  an  experiential  run  does  not  con¬ 
tain  the  optimization  of  the  expected  factors). 

The  details  of  these  six  steps  are  described  below. 


Identify  the  characteristic  to  be  observed  and  the  func¬ 
tions  to  be  optimized 

Traditionally,  friendly  military  forces  are  defined  as  “blue  force” 
and  enemy  forces  are  defined  as  “red  force.”  Primary  functions  for 
optimization  experimented  within  this  initial  trial  will  be  the  num¬ 
ber  of  blue  aircraft  lost  and  number  of  red  aircraft  destroyed.  The 
characteristic  to  be  observed  in  this  case  is  the  number  of  aircraft 


SIMPSON  &  UNAL  |  RSM  to  Optimize  Air  Power  9 


(Y  function).  The  objective  is  to  determine  the  optimum  combina¬ 
tions  of  design  parameter  values  to  minimize  the  number  of  blue 
aircraft  lost  and,  at  the  same  time,  maximize  the  number  of  red  air¬ 
craft  destroyed.  We  will  be  emulating,  as  best  we  can  quantitatively, 
Operation  Bolo. 


Identify  the  factors  and  levels 

In  this  study,  two  levels  of  each  parameter  were  studied:  a  high  (level- 
1)  and  a  low  (level-2)  value  (Unal,  Stanley,  and  Joyner  1993).  Factors 
and  values  of  the  four  variables  selected  for  study  are  in  Table  1 . 
Level- 1  factor  represents  units  that  are  fully  manned,  trained, 
equipped,  and  totally  capable  of  engaging  the  enemy.  Seventy  per¬ 
cent  was  selected  as  level-2  to  demonstrate  units  that  were  less  com¬ 
petent  but  still  an  effective  fighting  force. 

Table  1 .  Factors  and  levels 


Factors 

Level-1  (Yates  -1) 

Level  -2  (Yates  +1) 

A 

F18  Units 

100% 

70% 

B 

FI 6  Units 

100% 

70% 

C 

F15C  Units 

100% 

70% 

D 

F15E  Units 

100% 

70% 

The  levels  represent  an  outcome  that  a  commander  would  require 
to  be  studied  such  that,  for  various  combinations  of  parameters,  it 
would  remain  reasonable. 
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Define  the  most  likely  interactions  between  these  param¬ 
eters  (factors) 

Varying  several  factors  simultaneously  may  have  interactive  effects 
on  our  black  box  that  affects  the  optimum  solution.  When  the  effect 
of  one  parameter  depends  on  the  level  of  another,  an  interaction  is 
said  to  exist  (Kackar  1985).  It  is  important  to  understand  the  inter¬ 
actions  to  find  optimum  minimal  and  maximal  relations.  For  this 
operational  air  power  problem,  it  was  difficult  to  estimate  which 
pairs  of  parameters  will  have  the  strongest  interactions.  Therefore, 
we  investigate  all  interactions  that  may  be  significant. 


Design  the  matrix  experiment  required  and  define  the 
data  analysis  plan 

Using  Yates’s  algorithm  (Myers  1971)  to  code  the  experiments,  one 
would  expect  the  main  functions  to  react  as  depicted  in  Table  2  and 
the  interaction  functions  as  in  Table  3.  Instead  of  writing  each  num¬ 
ber  in  detail,  Yates’s  algorithm  allows  a  -1  to  indicate  high  level  and 
a  +1  to  indicate  a  low  level.  The  algorithm  requires  starting  with 
-1  and  then  alternating  to  a  +1.  Each  additional  column  of  factors 
requires  alternating  signs  in  pairs. 
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Table  2.  Main  Effects 


Main  Factors 

Experiment 

Number 

A 

B 

C 

D 

i 

-1 

-1 

-1 

-1 

2 

+  1 

-1 

-1 

-1 

3 

-1 

+  1 

-1 

-1 

4 

+  1 

+  1 

-1 

-1 

5 

-1 

-1 

+  1 

-1 

6 

+  1 

-1 

+  1 

-1 

7 

-1 

+  1 

+  1 

-1 

8 

+  1 

+  1 

+  1 

-1 

9 

-1 

-1 

-1 

+  1 

10 

+  1 

-1 

-1 

+  1 

11 

-1 

+  1 

-1 

+  1 

12 

+  1 

+  1 

-1 

+  1 

13 

-1 

-1 

+  1 

+  1 

14 

+  1 

-1 

+  1 

+  1 

15 

-1 

+  1 

+  1 

+  1 

16 

+  1 

+  1 

+  1 

+  1 
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Table  3.  Interaction  Effects 


Interactive  Factors 

Experiment 

Number 

AB 

AC 

AD 

BC 

BD 

CD 

1 

+  1 

+  1 

+  1 

+  1 

+  1 

+  1 

2 

-1 

-1 

-1 

+  1 

+  1 

+  1 

3 

-1 

+  1 

+  1 

-1 

-1 

+  1 

4 

+  1 

-1 

-1 

-1 

-1 

+  1 

5 

+  1 

-1 

+  1 

-1 

+  1 

-1 

6 

-1 

+  1 

-1 

-1 

+  1 

-1 

7 

-1 

-1 

+  1 

+  1 

-1 

-1 

8 

+  1 

+  1 

-1 

+  1 

-1 

-1 

9 

+  1 

+  1 

-1 

+  1 

-1 

-1 

10 

-1 

-1 

+  1 

+  1 

-1 

-1 

11 

-1 

+  1 

-1 

-1 

+  1 

-1 

12 

+  1 

-1 

+  1 

-1 

+  1 

-1 

13 

+  1 

-1 

-1 

-1 

-1 

+  1 

14 

-1 

+  1 

+  1 

-1 

-1 

+  1 

15 

-1 

-1 

-1 

+  1 

+  1 

+  1 

16 

+  1 

+  1 

+  1 

+  1 

+  1 

+  1 

The  data  analysis  approach  will  use  a  combination  of  response 
tables  and  regression  analysis  to  determine  the  interactions  of  the 
combatants.  Interactive  factors  are  assumed  to  be  bidirectional  and 
result  from  the  interaction  of  factors. 


Conduct  the  experiments 

The  results  of  the  1 6  experiments  are  presented  in  Table  4.  Complete 
records  of  the  experiments  are  available  in  Appendix  A. 
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Table  4.  Blue/Red  Aircraft  Lost 


Main  Factors 

Experiment  ^ 
Number 

B 

C  D 

Blue  Lost 

Y 

Red  Lost 

Y 

1  -1 

-1 

-1  -1 

36 

56 

2  1 

-1 

-1  -1 

39 

56 

3  -1 

1 

-1  -1 

35 

54 

4  1 

1 

-1  -1 

35 

54 

5  -1 

-1 

1  -1 

39 

56 

6  1 

-1 

1  -1 

34 

56 

7  -1 

1 

1  -1 

34 

53 

8  1 

1 

1  -1 

33 

52 

9  _1 

-1 

-1  1 

34 

56 

10  1 

-1 

-1  1 

35 

57 

11  -1 

1 

-1  1 

37 

53 

12  1 

1 

-1  1 

33 

55 

13  -1 

-1 

1  1 

33 

55 

14  1 

-1 

1  1 

33 

55 

15  -1 

1 

1  1 

35 

53 

16  1 

1 

1  1 

39 

55 

Analyze  the  data  to  determine  optimum  levels  of  factors 

Since  the  experimental  design  is  orthogonal,  it  is  possible  to  separate 
the  effect  of  each  parameter  (Bryne  and  Taguchi  1986).  The  aver¬ 
age  weights  for  each  factor  (as  explained  below)  were  calculated  and 
presented  in  the  following  response  tables. 
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Table  5.  Blue  Aircraft  Lost  Sensitivity  (R2  very  low) 


Factors 

A  (F18) 

B  (F16) 

C(F15C) 

D(F15E) 

Level- 1 

35.125 

35.125 

35 

34.875 

Level- 2 

35.375 

35.375 

35.5 

35.625 

Sensitivity 

-0.25 

-0.25 

-0.5 

-0.75 

Table  6.  Red  Aircraft  Destroyed  Sensitivity  (R2  High) 


Factors 

A  (F18) 

B  (F16) 

C(F15C) 

D(F15E) 

Level- 1 

55 

53.625 

54.375 

54.875 

Level-2 

54.5 

55.875 

55.125 

54.625 

Sensitivity 

0.5 

-2.25 

-0.75 

0.25 

The  response  tables  show  the  loss  of  aircraft  effects  of  the  factors  at 
each  level.  These  are  separate  effects  of  each  factor  commonly  called 
main  effects  (Phadke  1989).  The  average  aircraft  lost/destroyed  in 
the  response  table  are  calculated  by  taking  the  average  from  Table  4 
for  a  factor  at  a  given  level,  every  time  it  was  used.  As  an  example, 
the  factor  A  (FI  8s)  was  at  level-2  (annotation  in  table  4  being  [-1])  in 
experiments  1, 3,  5,  7,  9,  11,  13,  and  15.  The  average  of  blue  aircraft 
lost  is  shown  in  the  response  in  Table  5  under  A  at  level-2  (annota¬ 
tion  being  [+1]).  This  procedure  is  repeated  and  the  response  table 
is  completed  for  all  factors  at  each  level. 

The  number  of  aircraft  lost/ destroyed  effect  sensitivity  is  computed 
by  taking  the  difference  between  the  largest  and  smallest  number  for 
a  given  factor.  The  response  table  for  blue  aircraft  lost  reveals  that 
the  number  of  F 1 5E  shows  the  greatest  sensitivity,  meaning  that  the 
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largest  effect  on  blue  aircraft  lost  is  realized  by  varying  this  factor. 
The  response  table  for  our  maximization  problem  of  red  aircraft 
destroyed  reveals  that  number  of  FI 6,  factor  B,  shows  the  greatest 
sensitivity  (absolute  value  of  factor  B  level- 1  minus  factor  B  level-2, 
in  Table  6),  meaning  that  the  largest  effect  on  red  aircraft  destroyed 
is  realized  by  varying  this  factor.  Similarly,  blue  aircraft  lost  factor  A 
(FI 8s)  and  factor  B,  FI  6,  shows  the  least  sensitivity  (Table  5  factor  A 
and  B).  Additionally,  factor  D  (F15Es)  shows  the  least  sensitivity  to 
red  aircraft  destroyed  (Table  6).  The  average  aircraft  lost/destroyed 
for  blue  aircraft  lost  factor  A  (Table  5)  and  red  aircraft  destroyed 
factor  B  (Table  6)  are  also  graphed.  Blue  effects  were  “weak”  when 
compared  to  the  much  smaller  numbers  in  Table  5  as  compared  to 
the  -2.25  number  generated  in  Table  6. 


Blue  Aircraft  Lost  Sensitivity 


- F18  Factor  (Table  5) 


Figure  1.  FI 8  Sensitivity 
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Red  Aircraft  Destroyed  Sensitivity 


- F16  Factor  (Table  6) 


Figure  2.  FI 6  Sensitivity 


To  estimate  factor  interaction  effects,  a  two-way  interaction  response 
table  is  prepared  from  the  observed  data.  The  analysis  of  interac¬ 
tions  show  that  factor  B  (F 1 6)  X  1)  (F 1  oh)  interaction  for  blue  aircraft 
and  factor  A  (FI 8)  X  D  (F15E)  interactions  for  red  aircraft  exists. 
However,  these  interaction  effects  are  much  less  relative  to  the  strong 
if  (FI 6)  main  effect.  Using  the  same  procedure,  interaction  response 
tables  and  plots  can  be  used  to  analyze  all  other  factor  interactions 
to  be  studied.  Overall,  for  this  one  ATO,  interactive  effects  turned 
out  to  be  weak. 

The  optimum  levels  for  the  four  factors  can  now  be  selected  by 
choosing  the  level  to  minimize  or  maximize  to  reach  a  desired  out¬ 
come.  In  the  two  cases  investigated,  the  minimum  blue  aircraft  losses 
are  under  the  following  conditions: 
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Table  7.  Minimum  Blue  Loses 


A  1  F18  70% 

B  1  F16  70% 

C  1  F15C  70% 

D  -1  F15E  100% 


With  a  Y  (min)  of  Blue  aircraft  lost: 


Y  (min)  =  33.75 


Table  8.  Maximum  Red  Loses 

The  maximum  Red  aircraft  destroyed  under  the  following  conditions: 

A  -1  FI 8  -  100% 

B  1  FI 6 -70% 

C  1  F15C  -70  % 

D  -1  FOE -100% 


With  a  Y  (max)  Red  aircraft  destroyed: 


Y  (max)  =  56.75 

The  blue  losses  minimize  math  model  33.75,  which  is  within  the  low 
range  (33-34),  implying  we  can  minimize  FI 8,  FI 6,  and  F15C  and 
keep  F15E  at  full  strength.  The  red  losses  maximize  math  model 
56.75,  which  is  within  the  high  range  (56-57),  implying  we  can 
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minimize  FI 6  and  F15C  and  keep  F15E  and  FI 8  at  full  strength. 
The  operational  analysis  is  that  one  can  assign  F15C  and  FI 6  units 
some  slack  (i.e.,  a  30%  reduction  in  sorties)  in  flying  one  day  without 
major  impact  to  one  day  of  the  war  effort  (blend  Experiment  10 
and  11).  Those  results  are  based  on  the  linear  math  model  obtained 
using  RSM  design.  Linear  regression  provides  a  R2  value  allow¬ 
ing  the  determination  of  how  good  one  term  is  as  a  predictor  of 
another.  The  higher  value  of  R2  is  the  better  predictor.  The  model 
for  blue  lost  was  weak  as  it  had  a  low  R2  value.  This  result  indicated 
the  need  for  a  more  complex  math  model  and/ or  other  factors  not 
included  in  the  study  were  in  play.  The  math  model  for  red  aircraft 
lost  was  much  stronger  with  a  higher  R2  value  indicating  a  higher 
confidence  in  the  results. 


Conclusions 

One  of  the  goals  of  systems  engineering  is  optimization  beyond  a  sin¬ 
gle  node.  This  article  explored  a  quantitative  approach  (RSM)  in  an 
attempt  to  evaluate  a  complex  air  power  problem.  It  is  just  a  candle 
in  a  dark  night.  Further  research  should  be  undertaken  to  determine 
if  a  RSM  method  can  be  used  to  help  improve  the  establishment 
of  an  airhead,  to  better  understand  a  multitude  of  air-to-ground 
mission  issues,  or  even  to  maximize  something  like  Remote  Piloted 
Vehicles  (RPV)  coverage.  Different  operational  models  (other  than 
C2WSPTT)  and  math  models  (other  than  the  linear  model  used 
herein)  could  be  employed  as  comparison  to  historical  or  current 
operations  would  provide  a  venue  for  needed  exploration.  The  case 
studied  has  several  easily  recognized  limitations:  What  would  have 
happened  if  the  SAMs  were  left  on?  Are  the  unclassified  generic 
scenarios  realistic?  The  goal  of  the  article  was  to  explore  if  RSM 
would  work  on  an  air  power  problem,  not  to  authenticate  that  the 
air  power  problem  selected  as  the  backdrop  was  valid.  The  article 
described  using  RSM  in  the  1 2  hours  between  ATO  production  and 
implementation.  Including  other  parameters  overlooked  and  more 
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complex  non-linear  math  models  and  increasing  the  operational 
model  speed  and  number  of  required  runs  might  make  RSM  a 
costly  tool  to  be  used  in  Combat  Operations  during  execution. 

In  a  full  factorial  design,  all  possible  factors  and  levels  are  studied 
simultaneously.  Other  techniques,  such  as  Fractional  Factorial  or 
Central  Composite  Designs,  can  be  used  and  may  be  more  appropri¬ 
ate  for  the  factors  and  levels  studied.  As  the  goal  of  this  article  was  to 
explore  if  qualitative  engineering  could  be  used  within  an  Air  Power 
problem,  a  full  factorial  design  method  was  selected.  A  secondary 
goal  of  using  RSM  in  this  article  was  to  show  the  potentiality  to 
save  lives  exposed  to  risk  by  the  use  of  quantitative  tools.  In  the  real 
world,  it  most  likely  would  require  a  small  dedicated  team  trained 
in  these  techniques  to  study  the  problem  and  provide  a  quick  over¬ 
view  to  make  decisions.  The  authors  investigated  whether  a  qualita¬ 
tively  created  ATO  can  be  optimized  using  RSM  methods.  Given  an 
operational  model  that  is  a  valid  approximation  to  expected  reality, 
response  surface  methods  may  be  used  to  answer  specific  questions 
within  the  time  available  to  promulgate  those  answers  to  fielded 
forces.  Operation  Air  Power,  as  defined  by  the  ATO  produced  by  the 
AOC,  may  be  quantitatively  optimized,  providing  a  more  precise 
control  method  for  fielded  forces. 
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APPENDIX  A 

Only  units  that  lost  aircraft  in  the  referenced  experiment  are  listed. 
Fighter  Squadron  (FS) 

Squadron  (SQ) 

Marine  Fighter  Attack  Squadron  (VMFA) 

Navy  Fighter  Squadron  (VF) 


Experiment  1 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

4 

21  FS 

8 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

77  SQ. 

1 

24  FS 

4 

94  FS 

2 

33  FS 

6 

95  FS 

12 

42  FS 

2 

VF  103 

4 

51  FS 

6 

VMFA  513 

3 

52  FS 

6 

53  FS 

6 

Total 

34 

Total 

56 
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Experiment  2 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

4 

21  FS 

8 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

522  FS 

4 

24  FS 

4 

77  SQ, 

2 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

12 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

3 

53  FS 

6 

Total 

39 

Total 

56 

Experiment  3 


Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

2 

21  FS 

8 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

12 

24  FS 

4 

77  SQ, 

2 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

12 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

5 

53  FS 

6 

Total 

35 

Total 

54 
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Experiment  4 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

2 

21  FS 

8 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

12 

24  FS 

4 

77  SQ, 

2 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

12 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

5 

53  FS 

6 

Total 

35 

Total 

54 

Experiment  5 


Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

4 

21  FS 

8 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

522  FS 

4 

24  FS 

4 

77  SQ, 

2 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

12 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

3 

53  FS 

6 

Total 

39 

Total 

56 
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Experiment  6 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

4 

21  FS 

8 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

24  FS 

4 

77  SQ, 

1 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

12 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

3 

53  FS 

6 

Total 

34 

Total 

56 

Experiment  7 


Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

4 

21  FS 

5 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

24  FS 

4 

77  SQ, 

1 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

12 

51  FS 

6 

VF  103 

4 

52FS 

6 

VMFA513 

3 

53  FS 

6 

Total 

34 

Total 

53 
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Experiment  8 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

2 

21  FS 

8 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

10 

24  FS 

4 

77  SQ, 

2 

33  FS 

6 

42  FS 

2 

95  FS 

12 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

5 

53  FS 

6 

Total 

33 

Total 

52 

Experiment  9 


Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

4 

21  FS 

8 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

24  FS 

4 

77  SQ, 

1 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

12 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

3 

53  FS 

6 

Total 

34 

Total 

56 
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Experiment  10 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

6 

21  FS 

9 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

24  FS 

4 

77  SQ. 

2 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

10 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

3 

53  FS 

6 

Total 

35 

Total 

57 

Experiment  11 


Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

2 

21  FS 

9 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

12 

75  FS 

2 

24  FS 

4 

77  SQ, 

2 

33  FS 

6 

94  FS 

2 

42  FS 

0 

95  FS 

10 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

5 

53  FS 

6 

Total 

35 

Total 

53 
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Experiment  12 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

2 

21  FS 

9 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

12 

24  FS 

4 

77  SQ. 

2 

33  FS 

6 

94  FS 

2 

95  FS 

10 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

5 

53  FS 

6 

Total 

33 

Total 

53 

Experiment  13 


Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

2 

21  FS 

9 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

24  FS 

4 

77  SQ, 

2 

33  FS 

6 

94  FS 

2 

95  FS 

10 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

3 

53  FS 

6 

Total 

31 

Total 

55 
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Experiment  14 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

4 

21  FS 

9 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

14 

24  FS 

4 

77  SQ. 

2 

33  FS 

6 

94  FS 

2 

95  FS 

10 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

3 

53  FS 

6 

Total 

33 

Total 

55 

Experiment  15 


Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

2 

21  FS 

9 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

12 

24  FS 

4 

77  SQ, 

4 

33  FS 

6 

94  FS 

2 

95  FS 

10 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

5 

53  FS 

6 

Total 

35 

Total 

53 
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Experiment  16 

Blue  Unit 

Number  Killed 

Red  Unit 

Number  Killed 

14  FS 

4 

21  FS 

9 

78  FS 

4 

22  FS 

4 

27  FS 

4 

23  FS 

12 

75  FS 

2 

24  FS 

4 

77  SQ. 

4 

33  FS 

6 

94  FS 

2 

42  FS 

2 

95  FS 

10 

51  FS 

6 

VF  103 

4 

52  FS 

6 

VMFA513 

5 

53  FS 

6 

Total 


39 


Total 


55 


